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ABSTRACT 



This review of the Parent and Child Education (PACE) 



program discusses the professional literature that supports the PACE 
model r data analysis of the program, and suggestions for improving 
the design of program evaluation. The PACE program is t>ased on widely 
accepted ideas about the relationship between family characteristics 
and children's educational and social outcomes* An extensive 
literature shows that parental behaviors and attitudes are associated 
With children's vocational aspirations, and that these aspirations 
are related to educational achievement. Based on this literature, 
PACE hypothesized that an intergenerational cycle of undereducation 
in some families results in educational failure* The program hopes to 
change parental attitudes that undervalue education and to cause 
parents to serve as better educational role models and teachers of 
their children. Results of various investigations show that PACE met 
its initial objective of causing 70 percent or more of adult 
participants to obtain a GEO (General Educational Development) 
diploma, improve academics by two grade levels, or complete one 
program cycle. An implicit objective of changing parental aspirations 
for their children's educational outcomes was also met. However, the 
program needs an evaluation effort capable of demonstrating its 
effectiveness and cost-efficiency. Seventeen tables and charts and 74 
references are included. (RH) 
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EXECUTIVE SUMMARY 



This review of the Parent and Child Education (PACE) program provides 
a discussion of the professional literature that supports the PACE 
model, a discussion of past and recent data analysis about the 
program, and some suggestions for improvement in evaluation design. 

The PACE program is based on widely accepted ideas about the 
relationship between family characteristics and children's educational 
and social outccxnes. An extensive literature shows that parental 
behaviors and attitudes are associated with children's vocational 
aspirations, which are related to their educational achievement. 
Based on this literature PACE hypothesized that there is an 
"intergeneratlonal cycle of undereducation" In some families that 
results in children's educational failure. PACE sees this problem as 
a major contributor to Kentucky's status as a state with a low rate of 
high school completion and seeks to break this cycle by changing the 
family factors that are believed to perpetuate it. In particular PACE 
hopes to change parental attitudes that undezrvalue education and to 
cause parents to serve as better educational role models and teachers 
of their children. 

Initially PACE lacked funding for program evaluation, but program 
staff were concerned about demonstrating program effectiveness. They 
employed individuals from two programs at the University of Kentucky 
to perform evaluation studies. These studies had unavoidable but 
serious methodological problems that limit them as demonstrations of 
PACE effectiveness. These methodological problems are due principally 
to lack of comparison data to show how the same individuals would have 
fared had PACE not been available, as well as problems in measurement 
procedures. The studies do provide some descriptive and anecdotal 
Information about PACE. 

Analysis of information about the adult participants in the Department 
of Education's 240 system Adult Education data base provided 
additional descriptive measures of PACE performance and information 
about adult and child participants was also provided by PACE projects. 
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The results of these various investigations show that PACE met its 
initial objective of causing 70% or more of adult participants to 
obtain a GED, improve academics by two grade levels, or complete one 
program cycle, and the implicit objective of changing parents' 
aspirations for their children's educational outcomes. The studies do 
not evaluate other objectives of the program. The information 
available about some objectives seems to show that they have not been 
met, but this information is so flawed that it cannot be viewed as 
conclusive. The PACE-collected child data produced inconclusive 
results, probably because it is too early #o expect the program to 
have clear-cut effects on ^.ne children. 

The PACE program needs an evaluation effort capable of demonstrating 
its effectiveness and cost-efficiency. The design of this system must 
compare PACE outcc^es with expected outcc^es had PACE not been 
available to the same population. This requires that the outcomes of 
similar individuals involved In other programs be compared to those of 
PACE participants. The research literature on educational 
disadvantage demonstrates that PACE is a reasonable design to deal 
with well-known problems, but this does not guarantee that PACE 
services will actually result in the desired outcomes. Demonstration 
of the value of PACE requires that certain causal relationships 
between PACE services and outcc^es be demonstrated. 

We need to classify PACE as a "family support program" so evaluation 
efforts can address appropriate outcomes. It Is particularly important 
in thinking about PACE evaluation to remember that expected PACE 
outcomes will occur over a period of years due to changes in family 
factors related to achievement- Evaluation of the program on the 
basis of short-term indicators will not answer questions about its 
success. 

Adequate evaluation of PACE effectiveness requires that a series of 
research studies be conducted beginning this year. These studies 
would evaluate the effect of particular services included in the PACE 
program and the effect of PACE services taken together. These studies 
should compare PACE participants to similar individuals who were 
served by other programs or not served by any program. 

PACE is a reasonable design, but reasonable designs sometimes fail to 
accomplish their objectives. Since the PACE model Is being replicated 
across the country, Kentucky must demonstrate the value of the PACE 
model. This demonstration requires Improvements in program evaluation 
efforts. 
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Introduction 

The Kentucky General Assembly authorized the Parent and Child 
Education (PACE) program in 1986 because they believed that a 
significant number of Kentucky families suffer from an 
"Intergeneratlonal cycle of undereducatlon* caused by limited parental 
basic skills and attitudes toward education, limitations in 
children's learning skills, and limitations In parental child care 
skills. The Division of Adult and Community Education designed the 
PACE program to respond to these problems by providing Joint 
educational services to parents and children. PACE serves parents who 
lack a high school diploma or equivalency and their children aged 3 
and 4. Program services Include adult basic education for the 
parents. Instruction in parenting skills, early childhood education, 
and joint parent-child interaction and learning. PACE funds are 
available on a competitive basis to school systems in Kentucky 
counties where more than 50% of the adults lack a high school diploma 
or equivalency (Heberle, 1988). 

PACE has developed a reputation as a highly Innovative and effective 
program and has drawn Interest from diverse groups both in Kentucky 
and nationally. In addition to state funding provided by the General 
Assembly, the Ford Foundation and the Kennedy School of Government 
have funded certain dissemination activities beyond the scope of the 
current program, and the Kenan Family Charitable Trust has funded a 
projrct to replicate the PACE model outside of the Department of 
Education (Heberle, 1989). 

PACE staff have not made a formal statement of their underlying model, 
but informational materials about the program imply specific and 
elaborate causal relationships among faaiily educational history, 
family attitudes about education, chlldrearing practices, and the 
expected educational outcomes of children whose parents are poorly 
educated (Kentucky Department of Education, 1987a; Heberle, 1988, 
1989; Weiss, 1988). PACE materials imply two types of causal 
relationships. The first type of relationship which theorizes about 
the effect of family variables on children's educational outcomes will 
be labeled the "explanatory model". The second which makes statements 
abo'jt the expected benefits of PACE Instruction will be labeled the 
"prevention/remediation model". A description of these two models 
follows. 
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THE EXPLANATORY MODEL 



1. Individuals whose parents have negative attitudes and 
expectations about education are more likely to perform 
poorly in school than individuals whose parents do not have 
these negative attitudes. 

2. Children whose parents present poor educational role models 
are more likely to perform poorly in school than children 
who have good parental role models. 

3. ScMne parental styles are more likely to produce children who 
perform poorly in school than are others. 

4. Children who evidence deficiencies in learning skills in the 
preschool years are more likely to perform poorly in school 
than individuals who do not evidence these deficiencies. 



THE PREVENTION/REMEDIATION MODEL 

5. Positive changes in parental basic skills will improve 
children's school performance. 

6. Parental modeling of positive attitudes and behaviors about 
education will improve children's school performance. 

7. Positive changes in parenting skills will improve children's 
school performance. 

8- Improvements in preschool learning skills will improve 
children's school performance. 

9. Services delivered during a critical preschool time period 
will have a significant long-term effect on children's 
school performance. 

10. A combination of services provided simultaneously to parents 
and children will be more effective than presentation of the 
same services by different providers on different occasions. 

Following more than a year of program operation, PACE staff identilxe-i 
among other problems the initial lack of funds for evaluation. Since 
then they have turned to various post hoc evaluational procedures 
(Kim, 1987, 1989; Townley, 1987) which have failed to provide h clear 
analysis of the performance of the program. To remedy this problem 
the Division of Community Education requested that the Office of 
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Research and Planning assist them in summarizing existing information 
about the program and in designing an ongoing evaluation process to be 
incorporated into future PACE program services. 

This paper responds to that request. It addresses three different 
PACE program evaluation problems: 

1. The explanatory model is bc^sed on well-known relationships 
between adults' educational and social status cm J their 
children's educational and vocational attainment. PACF staff 
are familiar with the literature in these areas, but they 
have never explicitly summarized this literature as it 
relates to program design. Some of the research is so 
widely accepted that evaluation of these relationships in 
Kentuc)cy by the PACE program would be an unnecessary 
duplication of effort. The first section of this paper 
provides a summary of this literature along with a 
discussion of its relationship to the design and operation 
of the PACE program. 

2. At its inception the program had nine goals (Kentucky 
Department of Education, 1987b). Existing information does 
not show that all of these have been acccxQplished although 
information is available about some of them. 

Individuals associated with two programs at the University 
of Kentucky (Kim, 1987, 1989; Townley, 1987) have conducted 
three evaluation studies of PACE. These studies have 
unavoidable methodological flaws that call any causal 
conclusions into doubt, but they do provide some useful 
descriptive information. 

Additionally, the Division of Adult Education collects 
information about ABE students, including PACE students, and 
the Division of Community Education has collected 
information about PACE children and adults. Analysis of 
this information also provides Bomo descriptive information 
about program operation up to the present. 

The second part of this paper describes the PACE program and 
its participants based on these sources of information. 

3. PACE assumes that its approach is an improvement on 
traditional adult and early childhood programs because of 
the interactive effect of the various services provided. 
This is a causal statement that deserves an adequate 
evaluation. Public programs should demonstrate their cost 
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effectiveness (Barnett and Escobar, 1987), and PACE Implies 
that its approach will be more effective than other 
alternatives that might be provided to the same population. 
Since the prevention/remediation model requires a more 
thorough and methodologically adequate evaluation than has 
been done, the third part of this paper recommends a design 
for conducting this causal and cost-benefit analysis. 

A public program should have an evaluation system capable of 
monitoring program performance and isolating changes or 
problems that need to be addressed. The last section of 
this paper also addresses the Uesign of a routine data 
collection and analysis system for the PACE program. 
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Review of the Literature 



The PACE program Is neither an adult basic education program nor is 
it an early childhood program. Both types of programs attempt to 
overcome econcMnic and social disadvantage by providing compensatory 
services to individuals who suffer frcMn educational and/or cultural 
deprivation. Because PACE combines both t:ypes of services into a 
single program founded on well-known relationships between family 
background and children's educational and occupational achievement, it 
qualifies as a "family support program" (Denton, 1989; Weiss, 1989). 

PACE services are based on the belief that Illiteracy and certain 
other educational problems are due to an "Intergenerational cycle of 
undereducation" , i.e., that poorly educated individuals tend to 
produce children who are also poorly educated. A substantial tK}dy of 
research literature supports this belief. The relationship of 
parents' socioeconomic status to children's educational and 
occupational outcomes has long been known although the strength of the 
relationship has generally been modest. Recent investigations have 
demonstrated that when measures of socioeconomic status are 
supplemented with measures of family environment, the relationships 
become considerably stronger (Walbcrg & Marjoribanks, 1976; Bloom, 
1986). 

A number of investigators have demonstrated the general relationship 
between parental behavior and children's educational performance. 
Datcher-Loury (1988) found that the amount of time that mothers spend 
with their children is related to the number of years' schooling 
completed by the child but that this relationship occurs only for 
mothers with less than 12 years of schooling. Entwisle and Hayduk 
(1988) found in a follow-up of school children in Baltimore that 
parental influences at an early age had a substantial impact much 
later in the schooling process but their design did not allow them to 
evaluate the specific parental behaviors that caused these effects. 
McGowan and Johnson's (1984) evaluation of the performance of the PCDC 
program in Houston found that mothers' number of years of schooling 
was related to children's educational attainment but concluded that 
the specific relationships were too complex to be evaluated with their 
statistical model. Martin (1981) in a study of dropouts in urban and 
rural counties in Kentucky found 32 factors associated with dropout, 
including family educational background factors, child 
characteristics, and socioeconomic status. He did not report the 
strengths of these relationships. 

Other authors have demonstrated the strength of specific 
relationships. In an analysis of a 14-year longitudinal research 
project, Schiamberg and Chin (1987) found that youth occupational 
motivation was the best single predictor of children's educational and 
occupational attainment and that family background factors and child 



- 5 - 



>1 



characteristics are related to occupational motivation. Parental 
influence had an indirect effect on occupational attainment, mediated 
through the child's educational attainment. Marjoribanks (1984) found 
a relationship between parents' aspirations for their children and 
adolescents* occupational aspirations, in a study of Mexican-American 
children, Valencia and Henderson (1985) found that family background 
factors were the best predictors of children's educational 
performance. Laosa (1982) concluded frcxn a study of Hispanic and non- 
Hiiipanic white families that parents' strategies for teaching their 
children had a significant effect on children's academic outcomes; 
that this effect occurred for both Hispanic and xK>n--Hi8panic white 
families; and that use of specific teaching behaviors that were 
associated with children's performance was significantly related to 
parents' educational level in both ethnic groups. 

Specific parental behaviors as well as parents' educational status are 
related to children's educational outcomes. PACE intended to improve 
children's eventual academic and vocational success by providing 
parents with educational experiences that would change their behaviors 
to resemble those of families that encourage achievement. Th9se 
services were combined with an aarly childhood education prcjiam 
because early childhood programs have been shown to improve cMe 
educational attainment of economically disadvantaged children. 

Early childhood education has a long history as an attempted solution 
to the problem of social disadvantage. In a historical review Grubb 
(1987) traced the idea of early childhood Intervention back into the 
early I9th century, noting that it has undergone several changes of 
philosophy which reflect changes in beliefs about the causes of 
poverty. In recent years early childhood education was selected by 
the "War on Poverty" of the 1960 's as a means of compensating for 
social inequality (Jensen, 1985; Grubb, 1987) and more recently as a 
means of overcoming developmental lags due to childhood disability 
(Casto and Mastropierl, 1986). A number of controversies have arisen 
In this area, most often because of claims by proponents that early 
childhood programs could be expected to substantially increase the 
Intellectual capacity of children (Hoegl, 1985; Jensen, 1969, 1985). 
Despite these controversies early childhood programs are widely 
accepted as desirable and effective. 

A number of longitudinal studies have demonstrated the efficacy of 
early childhood program?. Of these, the best known and probably the 
best designed is the Perry Preschool Project (Berreuta-Clement , 
Schwelnhart, Barnett, Epstein, and Weikart, 1984). Researchers 
associated with this project have demonstrated cost benefits in the 
areas of greater high school graduation rates, lower crime rates, 
better occupational adjustment, better school peformance, and reduced 
dependence on welfare (Barnett, 1985). The project provided benefits 
to society, to attendees, and to taxpayers and potential crime 
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victims. Other longitudinal studies have demonstrated similar results 
for other early childhood projects (Bamett and Escobar 1987; Lally, 
Mangione, and Honig, 1987; Schorr, 1989). 

Early childhood programs vary widely in quality and methodology, and 
some types of programs may be more effective than others (Hebbeler, 
1985). Controversy has arisen over whether full-day services are 
essential and over different types of instructional styles (Bamett, 
Frede, Mobashar, and Mohr, 1987; Bereiter, 1986; Gersten, 1986; 
Schweinhart, Ifeikart, and Lamer, 1986). 

The PACE program staff used the High/Scope approach to early childhood 
education based on the Perry Preschool Project. While there is no 
guarantee that this is the best possible approach, they avoid some of 
the controversies by using a method that has demonstrated its 
effectiveness . 

The PACE program attempts to attack the problem of undereducation 
across generations in poor families with both adult basic education 
and early childhood services. This design responds directly to known 
relationships between family environment and children's educational 
achievement. PACE does not need to demonstrate the relationships of 
the explanatory model since literature in adult education, early 
childhood education, cwnpensatory educet-inn, and dropout prevention 
provides support for the strength of these relationships. 

That children's educational and vocational outcomes are associated 
with family background factors does not prove that these factors are 
the cause of the outcomes, nor does it guarantee that changes in 
family characteristics will cause changes in the outcomes. The 
prevention/remediation model is thus not validated by reference to the 
relationship between family background and outcomes because the 
causes of both the family background factors and the outcomes might be 
some third variable not evaluated in any of the previous research 
(Hayes, 1989). Questions about aspects of the prevention/remediation 
model that cannot be answered by the literature will be considered in 
the third section of this paper. 



Existing evaluation of the PACE program 

Efforts so far to evaluate the PACE program have been descriptive and 
anecdotal because the program lacked a built-in evaluation component. 
In this section I will discuss these previous efforts and discriminate 
between findings that tell us something useful about the program and 
those that are too methodologically weak to tell us much. In 
addition, information created from other data sources will bo reported. 

Previous evaluation studies 

Two studies In 1987 and one study in l^gs attelI^pted to demonstrate the 
effectiveness of the PACE program. All three studies encountered 
methodological difficulties that substantially reduce their validity. 
The researchers encountered problems that have been extensively 
discussed in the literature on educational and social science research 
design (Campbell and Stanley, 1963; Keppel, 1973; Keeves, 1988). 

The problems with these studies stem primarily frcwi their inability to 
eliminate alternative explanations for the outccmas of the PACE 
program. Evaluation of program effectiveness always requires a 
demonstration that the effects associated with the program could not 
have been due to some Irrelevant factor not under the program's 
control . The studies also suffer from very weak measurement 
methodology. It may have been that strong program effects existed but 
were not demonstrated because they could not be Isolated by the 
measurement methods. The 1987 Kim study for example used a 
questionnaire approach to evaluate whether parents had Improved in 
parenting skills, but these behaviors are probably better measured by 
direct observation. The Townley study attempted to use the McCarthy 
Scales to measure the developmental status of children even though 
both PACE and High/Scope philosophies reject this type of information 
as an outcome criterion. 

The studies do provide some descriptive information about the program 
which will be reviewed below. 

Evaluation Results I: The 1987 Yun Kim Study 

Yun K. Kim (1987), a graduate student at the University of Kentucky 
Human Development Institute, examined 77 Individuals who were 
currently enrolled in 6 of the PACE projects and 59 Adult Basic 
Education students who were enrolled In 5 non-PACE projects. Two of 
the six PACE projects had been closed for lack of participants, so the 
actual number of PACE sites In the study was four. She reviewed 
students' records and administered a questionnaire to both groups and 
an additional questionnaire to PACE participants. An exploratory data 
analysis produced the following results: 
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1. Th© mean age of adult participants in the two types of 
projects was roughly equivalent, but the modal age for ABE 
participants way 4 years younger than that of PACE 
participants. She concluded based on differences in the 
Interquartile range that the age distribution of ABE 
participants was much wider thari that of the PACE 
participants. 

2. The mean grade completed by participants in both groups was 
the ninth. ABE participants avei ged a larger number of 
years out of school than did PACE participants due to the 
inclusion of a few "outliers'* in the sample who had been 
out of school for more than 45 years. She did not reanalyze 
the data without these outliers. 



3. The racial composition of participants in both types of 
projects was predominantly white* 

4. PACE participants were almost exclusively (97.3%) female, 
while ABE participants included a much smaller proportion of 
females (59.3%). 

5. Almost all (97.3%) of the PACE participants were unemployed, 
while a smaller proportion (72.41%) of the ABE participants 
were unemployed. 

6. '•Informal assessment indicated that PACE participants 
typically had Incomes ''•..below the level set by the 
Department of Labor**. 

1. About twice as many (36*8% as opposed to 15.3%) of the PACE 
participants obtained a GED than did ABE participants • 
Follow-up indicated that the PACE program had met its goal 
of 70% GED completion, completion of one program cycle, or 
raising grade levels by two grades. 

8. The mean number of months* retention in the two programs is 
similar, but ABE students tend to drop out a bit earlier 
than PACE participants. 

9. The participants' descriptions of their long-range goals 
vary significantly between the two programs. 



10. Data from the PACE questionnaire generally indicated that 
the participants were very positive about the program and 
agreed that the goals of the project had been met. 



Because of its methodological problems this study should not be 
interpreted as a demonstration of the effectiveness of the FACE 



program. The obvious demographic differences between the control 
group and the PACE students could easily explain the different 
outcomes of the two programs. As noted above the study also suffers 
from Instrumentation problems. This study does provide some 
descriptive and anecdotal Information which will be discussed below. 

Evaluation Results II: The Kim Townley study 

Klra Townley (Townley, 1987) of the University of Kjntucky College of 
Home Economics, Department of Family Studies, performed an evaluation 
at one Harlan County PACE site in 1987. She Intended to address the 
children's services component of the program by administering the PACE 
parent questionnaire, a self -report by the parents of their impression 
of the effects of the PACE program on them and on their children. 

The Townley study is entirely a one-shot case study. No control group 
of any kind was used. In addition, as the author notes, she was able 
to Include participants from only one of several sites. Even if 
adequate controls were available, this problem would Impose serious 
restrictions on the generallzablllty of the evaluation results to 
other sites in the PACE program. 

In addition to the parent questionnaire, Townley administered the 
McCarthy Scales to some of the children in the project, with no 
comparison group and no pretest. She noted that this information was 
difficult to Interpret without control group data. Nonetheless she 
recommended certain curriculum Improvements, apparently because she 
assumed that a completely successful application of the PACE concept 
should produce children without rf.markable developmental weaknesses. 
Numerous other possible explanations exist for the weaknesses found on 
the McCarthy. For example, the children could have been severely 
development ally delayed on enrollment, and the PACE program could have 
caused improvement to a less-impaired level. In fact studies of early 
childhood programs over the pasr twenty years have demonstrated that 
children served in these programs improve but continue to function 
behind their non-disadvantaged peers (Jensen, 1985; Barnett and 
Escobar, 1987). Without comparison data this part of the study is 
uninterpretable • 

The Townley study provided some descriptive information about PACE 
participants, but this data is much weaker than the Kim studies 
because it is limited to participants in a single project. 
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Evaluation Results III: The Yun Kim 1989 Study 

In addition to the 1987 study, Kim (1989) conducted a much more 

extensive data analysis of the PACE program using much the same 

approach as her earlier study but without a comparison group. She 
found the following: 



1. 



2. 



3. 



4. 



5- 



6. 

7. 

8. 
9. 

10. 



PACE participants were 95% female, 
and 95% unemployed. 



98% white, 71% married. 



The mean age of adult participants was 28 years; the mean 
grade completed was the ninth; the mean number of years out 
of school was 11. 

The mean number of children living with the adult 
participants w ^ 2.5. Participants had an average of 3.8 
other family members who had also dropped out of school. 

The mean length of stay in the program was 6.38 months. 89% 
of participants agreed that they had achieved their goals 
for coming to the PACE program. 

44% of the adult participants were totally dependent on 
government sources of support, and they tended to come from 
families that traditionally had received one or more forms 
of government financial assistance. Of those supported at 
least in part by wages (53%), 20% were receiving food 
stamps. Thus more than half (64%) of the families were 
involved in one or more forms of public income support. 

The majority of the adult participants had spouses who had 
obtained high school degrees. Participants' paients had 
levels of education similar to their own. 

52V of the adult participants exited the PACE program 
because they had obtained a GED, and 29% stayed until the 
and of the program year without obtaining a GED. 



PACE adult participants appear to be somewhat 
demographically from the usual ABE student. 



different 



The proportion of PACE participants who had a desire to 
attend further educational programs declined from pretesting 
to posttesting. 

After program completion PACE participants had greater 
expectations that their children would complete high school 
and that they would attend a four year college program, but 



- 11 - 



ERIC 



they showed no change in their expectations that their 
children would attend a vocational school or two year 
college program. 

11. Adult participants at program cc»npleti on showed decreased 
expectations that they themselves wovXd attend college or 
vocational school . 

12. Both GED recipients and non GED recipients showed 
significant changes In TABE scores between pre- and 
posttesting with non GED recipients having significantly 
lower initial scores. 

13. There were inconsistencies in program administration between 
projects. 

14. A potential limitation on long-term program effects is 
imposed by the fact that adults who obtain their GED's are 
required to exit the program even if they have been enrolled 
such a short time that their children are unlikely to have 
derived much benefit. 

15. No change appeared to occur in the extent to which PACE 
participants communicated with school authorities about 
their children. 

16- The mean pre- and posttest TABE scores for PACE participants 
were as given below (Table 4-14 from Kim, 1989): 



Subtest pretest posttest 

Reading 7.86 9.03 

Mathematics 7.02 8.89 

Language 7.61 9.04 

Averaged Grade 7.26 9.02 



Evaluation results IV: Recent data analysis 

At the request of PACE program staff, projects submitted information 
in February 1989 on PACE adults and children and the siblings of PACE 
children. In addition, data frcMn the Department of Education 240 
system Adult Education data base and from the GED database was also 
analyzed. 

The median age of adult participants was 25 with a mean of 26.8. The 
maximum age was 54 and the minimum was 17. This information is 
portrayed graphically on page 13. 
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The median number of years out of school was 10 with a maximum of 38 
and a minimum of 1 . 

The median grade completed by adult participants was 9 with a maximum 
of 12 and a minimum of 2. 

The distribution of length of stay in PACE programs shows that there 
are probably two populations of PACE adults, those who stay briefly 
(four months or less) and those who stay longer. The former group 
consists largely of individuals with relatively high academic skills 
who quickly obtain a GED; the latter probably of individuals with 
lower skill levels who require more instruction. 



The Adult Education database describes 95% of the ?ACE adults as 
female and 98% as white. Two percent of the adults were described as 
handicapped. Table 1 shows the distribution of number of children by 
participant from this database. 



Table 1. 

Distribution of Number of Children 



Number of Children 

0 
1 
2 
3 
4 
5 
6 



Proportion 

.01 
.24 
.40 
.21 
.12 
.01 
.01 



Table 2 compares the status of PACE adults not currently enrolled who 
obtained GED's and did not obtain GED's, based on PACE- submitted data. 
Notable findings Include the following: 



1. Adults who achieved the GED are about as likely to be 
unemployed as adults who did not buV they are more likely to 
be looking for work. 



- 14 - 



2. Those obtaining a GED are more likely to attend further 
educational prograjtns* The proportion of these individuals 
however is small (about 8% of those obtaining a GED). 

3« Individuals who failed to obtain a GED are more likely to 
have moved from the area« 



Table 2. 

Adults Exited from PACE 
Statewide Totals 



Current Status of those not of those 

receiving GED receiving GED 





number 


% 




number 


% 




At home, not working 


67 


53. 


.6 


72 


50. 


.0 


Employed 


27 


21, 


.6 


30 


20. 


.7 


Seeking employment 


3 


2. 


.4 


17 


11. 


.7 


Enrolled in vocational 


1 


0. 


.8 


7 


4. 


.8 


school 














Enrolled in college 


0 




0 


5 


3. 


.5 


Enrolled in other ABE 


2 


1. 


.6 


0 




0 


On vocational school 


0 




0 


3 


2. 


.1 


waiting list 














Moved from area 


17 


13, 


.6 


4 


2. 


.7 


Current status unknown 


8 


6, 


.4 


3 


2. 


. 1 


Total 


125 






145 







The absence of a comparison group makes it impossible to interpret the 
significance of these findings. The Division of Adult Basic 
Education does not conduct routine follow-up studies of individuals it 
has served, and as noted above PACE students are not similar 
demographically to individuals served by other ABE programs. 

Another problem of interpretation is that logically both PACE and 
adult education are intended to have long-term effects, and only a 
short time has elapsed since these adults exited frcMn FACE projects. 
Since most of these are young women with small children it should not 
be too surprising that they have not immediately obtained work or 
enrolled in college. That a small but significant number of them are 
enrolled in further educational efforts or looking for work is 
evidence of an immediate effect that might become a long-term trend. 



Table 3a gives the pre- and posttest academic levels for PACE students 
for reading, math, and language, and the mean differences for 
individuals' pre- and posttest scores. These levels are similar to 
those found by Kim (1989), but note the cautions below about the use 
of change scores. 

Table 3b shows the cumulative distribution of grade level scores for 
adult PACE students at pre- and posttest for reading and math. These 
scores are the pre- and posttest academic skills rounded to the 
nearest grade level. Twenty- two percent of the PACE adults gained two 
or more levels in reading; 35% gained two or more levels in math; 
and 34% gained two or more levels in language. 



Table 3a. 







Entry 


Exit 


Mean Difference 








Mean 


Mean 






Reading 




8. 


3 


9.3 


1.1 




Math 




7. 


2 


9.0 


1.7 




Language 




7. 


7 


9.1 


1.1 












Table 3b. 












Cumulative Distribution 




Academic 












Math Post 


Level 


Reading 


Pre 




Reading Post 


Math Pre 


1 


.00 












3 


.02 






.01 


.00 




4 


.05 






.02 


.03 


-Ox 


5 


.13 






.07 


.13 


.06 


6 


.20 






.12 


.36 


.12 


7 


.36 






.20 


.61 


.22 


8 


.49 






.30 


.80 


.35 


9 


.66 






.43 


.93 


.57 


10 


.88 






.63 


.98 


.74 


11 


.92 






.80 


.99 


.82 


12 


.96 






.85 


.99 


-85 


13 


1.00 






1.00 


1.00 


1.00 
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Table 4 shows the distribution of exited individuals obtaining a GED 
by site, based on the PACE-submitted data. The available information 
does not permit an analysis of the large differences among the various 
sites on this variable. 



Table 4. 



Site Location Number Obtaining GED Proportion 



Estill 


10 


.77 


Magoffin 


7 


.70 


Harlan 


29 


.63 


Monroe 


5 


.56 


Fleming 


5 


.56 


Spencer 


6 


.55 


Butler 


6 


.50 


Metcalfe 


11 


.50 


Bell 


17 


.43 


Hart 


14 


.41 


Letcher 


9 


.38 


Owsley 


3 


.25 



Of 282 adults exited from the program, 147 or 52.1% were reported by 
the PACE data to have obtained a GED. The GED data base shows that of 
310 former PACE students who took the GED test 222 or 72% passed, 76 
or 24% failed, and 12 or 4% were incomplete. The total enrollment in 
PACE between program startup and January 1989 was 565 individuals. 
Due to defects in the Adult Education and PACE datasets, we cannot 
give a precise number for the individuals who have exited the program. 
The 222 individuals who have completed a GED represent 39% of those 
who have been enrolled, but this underestimates the GED rate since we 
should not Judge the success of the PACE program on the basis of 
whether Individuals still enrolled have received a GED. Results from 
the two databases seem discrepant and we are unable to resolve these 
discrepancies with the available information. 



Tables 5a and 5b give the distribution of exit reasons for PACE 
students from the PACE-submitted and Adult Education data 
respectively. The categories used by the two datasets are not exactly 
equivalent but seem roughly similar. 
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Table 5a. 



( PACE-subraitted ) 



Exit Reason 


87 


88 


Total 




ou 




by 


End of program year 




32 


32 


Moved 




6 


6 


Other reasons 


1 


5 


6 


Got a job 




4 


4 


Fa i 1 Ad 




O 


o 


Lack of Interest 


1 


1 


2 


Parent still enrolled with younger child 




2 


2 


Family problem 




2 


2 


Child aged out 


1 




1 


Table 5b • 








(Adult Education data base) 








Exit Reason 


87 


88 


Total 


Completed their objectives 


11 


105 


116 


End of program year 


39 


81 


120 


Health reasons 


1 


5 


6 


Child care/family problems 


12 


18 


30 


Class time/ location or moved 


3 


13 


16 


Lack of interest 


1 


9 


10 


other known reason 


3 


7 


10 


Unknown reasons 


3 


9 


12 



Table 6 gives the current enrollment status of PACE children as given 
in the PACE-submitted data. The majority of those children for whom 
current enrollment information was reported are currently enrolled in 
kindergarten. Information about the attendance of these children was 
also submitted. This information Indicates that the PACE children 
when they entered school were absent an average of .81 days per month. 
71% of the children missed no days of school, with the other 29% 
accounting for all of the days missed. Of the 90 children for whom 
kindergarten or first grade special services referral information was 
available, nine hp.d been referred for speech and language services, 
three had been referred for remediation, and one had been referred to 
special education. 
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Table 6. 

Former PACE children current enrollment 

None 28 
Kindergarten 82 
First Grade 6 
Headstart 3 

In the absence of control group information. Table 6 tells us little 
about the success of PACE. We do not Icnow how often a similar child 
is absent from school or how likely s/he is to be referred for special 
program services. Also keep in mind that those data are mostly from 
very early in the school careers of these children before there has 
bean time to establish much of a trend. 

Table 7 gives the income sources for PACE adults as given in the 
Adult Education data base. These numbers are similar to Kim's data in 
that they show that more than 90% of PACE adults are not employed at 
program entry. The majority of the participants are described as 
having no income. This might include those Individuals described by 
the Kim data as deriving their living from their spouse's wages. The 
Adult Education data described 88% of PACE participants as below the 
poverty level based on the Department of Labor income status level. 



Table 7. 

Income Source of Adult PACE participants 



Income Source 


Number 




% 


Job or self-employed 


31 


8 


.3 


Job training or work-experience program 


1 


0 


.3 


Unemployment compensation 


3 


0 


.8 


Welfare, ADC, Disability, Other public 


102 


27 


.4 


Social Security 


11 


3 


.0 


Other source 


31 


8 


.3 


No income 


194 


52 


.0 



Program staff felt that within-family comparison of absence and 
retention statistics would demonstrate a PACE effect if it were shcvn 
that children enrolled in the PACE program were less likely to be 
absent, referred for special programs, or retained than were their 
older siblings who had not been enrolled in PACE. Information 
submitted on 57 siblings of PACE children shows the siblings missed an 
average of 8.9 days per year, and that 29% of them were retained in at 
least one grade. The PACE child data shows the kindergarten and first 
grade former PACE children missed an average of .81 days per month. 
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PACE Children's absences are given in months rather than years because 
most of the former PACE children are in their first (kindergarten) 
year of school- The two figures do not look very much different, but 
it is really too early to toll whether a difference exists. We would 
also expect (given the PACE prevention/remediation model) that the 
siblings* school performance would improve. Demonstration of the 
success of the PACE program In this area requires a reduction in 
absences and referral to special programs by the siblings of PACE 
children, and an even greater reduction in these areas by former PACE 
children. 
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Conclusions 

Since the PA-E program is now in its third year of operation, program 
staff by now have responded to those problems that can be identified 
on an anecdotal basis. The attempts at quantitative evaluation have 
been troublesome because they were conducted after the fact when it is 
very difficult to avoid serious methodological errors. The available 
results indicate that scHne of the original goals of the program have 
been met, but we cannot show that PACE is doing anything that other 
programs could not do. In the long run PACE must show this to justify 
its existence. 

The explanatory model establishes the appropriateness in theory of the 
PACE concept as an intervention for individuals who are \indereducated. 
The literature in this area suggests that undereducation is a family 
phenomenon. if one accepts this literature as demonstrating causal 
relationships between family factors and children's outccnnes, it is 
reasonable to believe that a program that addresses these family 
factors could ameliorate their effects on children's educational 
outcomes (Denton, 1989). PACE is an appropriate design to deal with 
these problems. 

The problem for PACE is that its existing evaluation capacity and 
procedures cannot demonstrate that it is effective. Programs based on 
well-founded designs sometimes do go astray despite the best of 
intentions and in the absence of an evaluation system it is impossible 
to determine that this has occurred. Even when a statewide program 
such as PACE is generally successful, there is usually some variation 
among the performance of particular sites, and administration of the 
program demands that a more detailed view of the various sites be 
available. PACE needs to develop an ongoing evaluation system, 
including a demonstration of its cost-effectiveness and cost-benefit 
as well as routine monitoring reports. 

The following specific conclusions are supported by the available 
evaluation information: 

A. Achievement of PACE goals. 

PACE originally established nine goals. Some have been met, some 
have not, and others have not been evaluated, as described below: 

1. Raise the educational levc*l of the parents of preschool 
children through instruction in basic skills 

It seems clear that this goal has been accomplished to some 
extent. The Kim 1989 study showed that adults enrolled in 
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PACE projects had Increased an average of 1.17 grade levels 
In reading, 1.87 grade levels in mathematics, and 1.43 grade 
levels in language as measured by the Tests of Adult Basic 
Education. She did not report statistical tests of these 
differences, but these are obviously significant. PACE has 
increased the educational levels of adult participants to 
some extent. 

One of the measurable objectives given by the program was 
that 70 percent of parents would successfully complete the 
GED test and/or raise skill levels on the 1ABE and/or enroll 
in further educational or Job training programs. This 
objective has been accc^plished as reflected both in the Kim 
and Division of Community Education data. Note that this 
result is due to the program's number of GED's and retention 
information alone. The data as reported do not support an 
unduplicated analysis of the participants' gain in academic 
skills. 

Enhance parenting skills 

The PACE program established a clear standard for success in 
the adult basic education comsK^nent (obtaining a GED, 
improving by two grade levels, or enrolling in further 
education) and used a standard curriculum with established 
outcome expectations for the child component. The program 
established no such standard for parenting skills. 
Background materials about the program identify enhancement 
of parenting skills as a progr';Am goal but fail to establish 
a model for adequate functioning as a parent. 

Kim's 1989 study attempted to evaluate this issue by asking 
parents about changes in their approach to disciplining 
their children. Since the model to which this information 
relates is unclear, the results are inconclusive. Townley 
asked parents how their responses to their children had 
changed but also did not seem to have a particular model 
of effective parental behavior in mind. She found changes 
in how parents deal with their children, but these cannot be 
evaluated in light of this failure to specify a model. Keep 
in mind as well that this result occurred at a single site, 
and that the measurement procedure (survey data) could have 
been contaminated by participants' knowledge of the 
expectations of project staff. 
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Enhancement of parenting skills is an important element of 
the explanatory model because specific parental behaviors 
have been shown to be related to children's educational 
outcomes* Any theoretical statement about this population's 
achievement levels or PACE*s value as an effective remedial 
effort must address this relationship (Hayes, 1989). If the 
PACE program continues to pursue this goal it should specify 
the changes in parenting skills that are desirable and 
should measure their change over the course of program 
participation « 

The researchers measured one parental behavior knovm to be 
related to chil.dren*s educational performance (reading to 
children) and produced equivocal results* This could ^ave 
been either because the program was ineffective or beci»ase 
the instrumentation and data collection were faulty. Kim*s 
attempt: to measure this beiisvior used two dif fe' ent 
indicators on two occasions. The alternative hypotheses .ae 
to such methodological errors need to be eliminated in 
future analysis of this issue. Even had an adequate 
evaluation of changes in this behavior been done, however, 
the lack of a theoretical framework would have obviated any 
conclusions thai; could be drawn. ''Reading to children" 
might be a cause of their educational outc<Miie8, or it might 
be an Indicator of a deeper level of relationship between 
parental behavior and the outcomes. Only a clearly 
elaborated theoretical model can make It possible to test 
such ideas. 

Increase developmental skills of preschool children to 
better prepare them for academic success 

No adequate analysis of this issue has been done. Kim 
collected information in this area but it was all survey 
data and probably was too weak to tell us anything for 
certain. No pretest was given. Townley used the McCarthy 
Scales to evaluate this issue but noted that the absence of 
either pretest or comparison group data did not allow her to 
show that Improvement had taken place. An analysis of Child 
Observation Record pre- and posttest scores done in 1987 
showed an overall 28% Improvement in chlldrens^ scores. 
Since this Improvement could have been due to maturation of 
the children, the results cannot be interpreted as a 
specific program effect. 
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Further evaluations of PACE need to specify the particular 
developmental skills of interest and the instruments to 
measure them. All children need to be both pre- and 
posttested, and a baser ate of change in these skills due to 
maturation needs to be established either through use of an 
instrument with known norms or through the use of a control 
group. 

Enable parents to becc^e familiar with and comfortable in 
the school setting. 

No attempt was made to directly evaluate this goal of the 
program. 

Provide a role model for the child of parental interest in 
education 

PACE materials make It clear that the program's primary 
indicator of parental educational role-modeling is whether 
the parent obtains a GED. Since more than half of the 
adults enrolled in PACE projects eventually do obtain a GED, 
one can say that this goal has been substantially met. 

Equating parental role-modeling with obtaining a GED Ignores 
a substantial body of literature that shows that parents' 
attitudes about education and educational aspirations for 
their children are strongly related to children's academic 
and vocational outcomes. Inclusion of these other elements 
of parental role-modeling in PACE evaluation would do a 
better job of demonstrating the program's effectiveness. 

PACE staff see participation by parents in an educational 
program as modeling behavior. It would be possible to 
measure the effect of this variable on the children. The 
effect of this and other specific parental behaviors on 
children's thinking and performance need to be measured if 
an adequate evaluation of PACE is to be done. 

The measurable objective of improving parental willingness 
to communicate with school personnel and personnel of other 
social service agencies fits here to scmbs extent. Program 
efforts to measure this objective showed no Improvement, but 
the measurement methodology was very weak. 
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Enhance the relationship of the parent and child through 
planned structured interaction 

As with the program's goal of enhancing parenting skills the 
major problem with evaluating this goal is that PACE has not 
established clear expectations for what is to be 
accomplished. Part of the PACE program is joint Interaction 
and learning between parents and children and the 
prevention/remediation model hypothesizes that this is one 
of the important interactive effects that sets PACE apart 
from other programs. Survey Information from the Kim and 
Towniey studies attempts to address this area with 
Inconclusive results because of methodological problems 
mentioned above. PACE must specify what is to be 
accomplished in this area before this important goal of the 
program can be evaluated. 

Demonstrate to parents their power to affect their child's 
ability to learn 

No effort was made to directly evaluate this goal. 

Encourage early identification and treatment of physical or 
mental handicaps that may inhibit the children's learning 
ability 

No effort was made to directly evaluate thic ^oal. 

Encourage identification and creatment of any handicapping 
condition in the adults that may inhibit their ability to 
care for their children 

The Adult Education database identifies 2% of the PACE 
adults as handicapped, but does not support an analysis of 
disability types. The University of Louisville Urban 
Studies Center estimates that in the age range 16-64, 14.1% 
of the population of Kentucky have a handicapping condition 
(Weir, Bonham, & Hunter-Manns, 1989). The 2% reported by 
the database is certainly signif ic<intly different from this 
figure, but this could be due either to recruitment from a 
population with a low Incidence of handicaps or to a low 
rate of reporting by project staff. In either case this 
information tells us nothing about the achievement of this 
goal because identif icstion of individuals with known 
handicapping conditions is not the same as encouraging the 
Identification and treatment of conditions not previously 
Identified or treated. 
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B. Description of the population served. 

A substantial proportion of the PACE adult participants derive 
some or all of their income from government programs and nearly 
half derive no income from either their ovm or their spouse's 
wages. The Adult Education data show that the majority of PACE 
adults live below the federal poverty level. 

Kira'f! analysis of TABE scores placed mean pretost scores for 
reading, mathematics, and language for PACK participants at about 
thf> seventh-grade level. Some individuals involved in the 
prr^ram had scores much lower than this but PACE students 
generally had relatively high academic skills pretest scores. 
Pretest scores from the Adult Education database placed 80% of 
the participants* reading pretest scores at or above the seventh 
grade level. The distribution of mathematics pretest scores 
placed the particpants at lower levels in this area than in 
reading . 

Information from both the Adult Education and Kim data shows that 
PACE participants are typically young white women who dropped out 
early in high school and have been out of school for several 
years . 

Kim's analysis that the spouses of a majority of PACE 
participants had high school diplomas or equivalencies confuses 
the picture somewhat. PACE assumes that an intergenerational 
cycle of undereducation exists, caused by parental negative 
attitudes about education, reflected in part by failure to obtain 
a high school degree. If one of the parents has a high school 
degree, some of this intergenerational devaluation of education 
might have been ameliorated. 

C. The program's effect on its participants 

The 1989 Kim study does provide some information about positive 
changes in adult participants. The m.-^st Important and apparently 
the most reliable of these is that the participants' expectations 
for their children's educational outcomes (finishing high school 
and attending college) Increased from pretest to posttest. This 
is almost certainly a true change in parental attitude and 
probably is due to program participation (although it could be a 
general effect which would occur in any adult education program). 
A related and problemmatic finding is that parents apparently 
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reduced their expectations about their own possible rurther 
educational accomplishments. Kim assessed this as implying that 
the adult participants became more realistic about their 
educational possibilities. This is plausible, but there is no 
really good reason to accept her interpretation. About all that 
can be said is that on posttest the participants were less likely 
to endorse college or vocational school as further educational 
possibilities for themselves. 

This one result demonstrates that the PACE program is effective 
in causing change in at least one of the areas central to the 
explanatory model. The literature review demonstrated thai 
parental aspirations for their childr n are related to children's 
academic outcomes and this is evidence that PACE has caused a 
change in parental aspirations. 

A better model for the "cycle of undereducation'* . 

It is necessary to clarify a few issues related to the logic of 
PACE'S theory about how the "cycle of undereducation" relates to 
program outcomes. As noted above, there is sufficient reason to 
believe that this problem exists. It is however important to 
keep a few things in mind. 

1. The definition of "undereducation" has changed considerably 
in recent years and continues to do so. The idea of high 
school graduation as a standard of educational success is of 
relatively recent origin. Until the 1940 Census the 
standard of educational performance was literacy, i.e. 
whether an adult could read and write (U.S. Bureau of the 
Census, 1943). in the 1940 and subsequent Censuses the 
definition became the number of years of school completed, 
and it was not until the 1970 census (U.S. Bureau of the 
Census, 1973) that the proportion of high school graduates 
was reported as a separate statistic. 

Within this framework significant changes have occurred in 
the educational status of the population both in Kentucky 
and nationally. This chart shows the median grade completed 
by individuals in Kentucky and the nation for the Census 
years 1940-1980. During this period the proportion of 
Kentuckians over age 25 wno were high school graduates rose 
from 15.5% to 57.1%. 



- 27 - 



8.3 



Kcntwcky 



7.5 



1940 1950 1**0 1970 



The implications of this Information help clarify the 
problem addressed by PACE. Educational levels In Kentucky 
and in the nation have risen steadily throughout this 
century, and graduation fr<Mn high school is now considered a 
minimum educational outcome. Rising expectations about the 
edticational levels of individuals have resulted from changes 
li; the complexity of infox-matlon skills required to 
participate and work in an increasingly technological 
society (Chall, 1987; Whirabey, 1987). This progression of 
expectations implies that families with an 
"intergenerational cycle of undereducation" are those whose 
members persistently achieve less than is educationally 
expectable for their generation, because they do not share 
values about education of others in their culture. 

Identification of the parents and grandparents of these 
individuals as high school dropouts is not a very good 
measure of this intergenerational pattern because until 
recently the majority of adults at most socioeconomic levels 
would have had similar family histories. It Is necessary in 
Judging family patterns of undereducation to compare the 
educational attainment of individuals with the expected 
achievement of the generation of which they are a member. 

The understandable tendency to look for immediate indicators 
of program success is counterproductive to operation of the 
PACE program. The focus on the Intergenerational cycle 
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Implies that the principal changes intended by the program 
will occur over a lengthy period of time. The usual 
standards of success for adults in ABE programs are not 
necessary to these changes except as they represent 
intermediate steps in the process. PACE should of course 
attempt to obtain as many GED's as possible for its adult 
participants, but obtaining a GED need not be the sole or 
even a major outcome of the program. The ultimate 
evaluation of PACE's success must depend on its ability to 
show that the children over their school lives are less 
likely to be undereducated than they would be if PACE were 
not provided. 
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Reconimendatlons 

Develop an ongoing management information ^nd evaluation system - 

PACE has had some problems because of a lack of an organized data 
collection and analysis system • This type of system is essential 
to the management of any human services prorvam, and it is 
recommended that PACE develop such a system. A substantial part 
of the system could easily be developed from existing sources of 
information* The Division of Adult Education collects an 
extensive data set on each individual served, and PACE projects 
are part of this system « A routine analysis of this data would 
provide the PACE program with an ongoing management information 
and program evaluation system. Because of the special nature of 
the PACE program^ some additional information needs to be 
collected. This information includes: 

Parent and child public program participation 

Child^s previous enrollment in educational programs 

Parent questionnaire data 

Child educational skills measures 

Child* s handicap status 

GED completion status 

Child follow-up information 

Number absences in primary grades 

Promotion/retention status 

Academic achievement (grades and test scores) 
Referral to special programming 
Parent follow-up information 
Work status 

Further educational enrollment 

Routine reports might include: 

Demographic der^criptions of participants 
Income levels 

Age, gender, educational history 

Handicap/health status 
Relationship of PACE to other programs 

Number of participants also served by each program 

Referral patterns frcxn other programs 
PACE outcomes 

Number of individuals achieving GED^s 

Children's school entrance and progress 

These are just examples: many other reports would be supported 
by the data and it would not be difficult to develop a system 
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of these 

f would be hi 

statistics "^11 a 

identifying and resolving program problems. 

^ « 4>s*4- PACE nrolects have sometimes been less than 
atligir in's^pply^ng InforSatl^n'lor the adult education database. 
Pr^l^ mon^tor^ng^'in this area should be improved. 

2 Define the size and location of the service population. 

r?S:iS^ lUlT^^Tj^oTlJ'tZllt. ^ralte^S^neS 
ir^trse'^nSI; w^rl^^^ ^^^^^ 

available. 

3. Define the socioeconomic characteristics of the population. 

rJoflnt-intion of the socioeconomic characteristics of the 
A good description or yne °^ -asantial to the eventual 

and as an ongoing program evaluation effort. 

4, Determine the best siting for the program. 

The General Assembly -quired that PACE services be ^^^^^^ 

through local P^^^^^^/^^^i^' ^cSSol context w!ll contribute to 
services provided in the Public ^^Jo evidence exists showing that 

IZ'^^ll^lT'l'^t^^''^^^^^^ nil 

?^:;;!ic??rb^thf^"-oStd dU^ln^'lt^lZr this is rean, the 
best approach. 

- I^^^fic^^V ^rMner-s.if?^^rrSc«^idufatiora! 
achievement 

^ . , J use of "readiness skills" 
rd*'1Srusr^f''afade^-Sriff ar»rsur^ of progra. outco.es 
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on the grounds that the measurement of academic skills at these 
early ages is not predictive of the eventual performance of 
children. This adds an additional hypothesis to the 
prevent ion/ remediation model, that measured readiness skills are 
less predictive of eventual school performance than attendence in 
a preschool program. This requires a longitudinal design with 
children in a direct-instructional group as a control, and with a 
nontreatment control. 

6. Reconsider the program objectives. 

The nine program objectives established at the beginning of the 
program should be reconsidered. Some of them are central to the 
programme mission (e.g., raising parental educational levels), 
but others are incidental. One reason why evaluation efforts 
have been so thorny is that program staff did not give enough 
attention early in the programme history to the objectives, how 
they could be acccmplished, and how they could be measured. it 
is understandable that at the beginning of a new program there 
would be some fuzziness in the establishment of objectives. The 
program has enough experience at this point however to allow some 
decisions to be made about its future standards. 

7. Improve PACE - Headstart interfacing. 

Headstart staff report that they have waiting lists for their 
services and children who are enrolled in PACE are not continued 
on Headstart waiting lists. If the PACE child leaves the project 
for some reason s/he often does not immediately re-enter 
Headstart, but instead re-enters the waiting list. This may 
result in the child losing any chance of re-entering Headstart. 
This is significant in light of the fact that program services 
are often terminated when the parent obtains a GED, even if 
parent and child have been enrolled only briefly. 

8. Demonstrate the causal links in the prevention/remediation model. 

PACE generated an elaborate — if scxnetimes implicit — model 
that predicts that certain remedial and preventive services 
should result in Improvements In the educational and social 
outcomes of disadvantaged children. This model Is based on well- 
accepted beliefs about a causal relationship between family 
constellation factors and child outcomes, from a literature that 
shows relationships between certain family patterns and outcomes. 
PACE theory is based on the assumption that the relationships 
exist because family patterns are the cause of the outccnnes even 
though the relationships do not constitute prima facie evidence 
of causality. Thus, PACE services are intended to change the 
outcomes by changing the family patterns. 
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Because the causal relationships have not been proven, doubt 
exists about the expected effectiveness of the PACE program • If 
it were true that the family factors and the outcomes were both 
the result of some unseen third factor, then changing the family 
factors would be unlikely to change the outcomes. In this case 
PACE would have no effect even though it is well-founded in 
existing research literature. For this reason it is important to 
test the hypotheses of the prevention/remediation model. If 
these tests demonstrate a program effect, then substantial 
support will be given to the causality of the relationships in 
the explanatory model. 

A design of a research program to test the prevention/remediation 
model is given in the next section. 
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DEVELOPMENT OF A RESEARCH DESIGN 



The PACE program suggests as a major cause of undereducation that 
children are affected by the educationally-related attitudes and 
behaviors of their parents. Substantial research supports this view 
and to the extent that the program Is deslgi ed to cause changes In 
these attitudes and behaviors. It approprl t ,ly addresses the problem 
of educational disadvantage. 

The appropriateness of the PACE model In theory does not guarantee 
that it will have predictable positive effects in practice at 
different sites administered by different organizations. The adult 
basic education, early childhood, and family service areas accomodate 
many possible program designs, some of which existed prior to PACE. It 
has not been shown that family constellation factors and parental 
behaviors are the causes of undereducation, only that a relationship 
exists. In the long run the value of PACE depends on showing that it 
accomplishes in practice what theory predicts and that it is a more 
desirable alternative for the appropriate population than other 
equivalent options that might be pursued with the same funding. This 
requires at minimum that PACE establish clear expectations for what 
participant behaviors and abilities are to be modified by program 
services . 

Demonstration of the value of an intervention requires that its 
outcomes be compared to what would have happened if no intervention 
had taken place. We cannot make these comparisons on the basis of 
current evaluations of the PACE program* 

Kim's comparison of information about non-PACE Adult Basic Education 
students with information about PACE students failed to solve thi^ 
problem (Kim, 1987) because the two populations are quite different* 
As her study shows, ABE participants were much more likely to be 
employed than PACE participants and were much less likely to be 
female. Given that PACE programs provide services that are not 
provided by ABE programs (early childhood education, lunch^ 
transportation) and the requirement that PACE participants have 3 or 4 
year old children, the motivations for enrollment of ABE participants 
were probably quite different frc»i those of PACE participants. Kim 
notes that the ABE group included a "few outliers'' who had been out of 
school for extended periods of tlme« Outliers are usually individuals 
from populations other than the one of interest (Snedecor and Cochran, 
1976). Since the ABE data probably Includes Infoxrmation from at least 
two distinct populations, the interpretation of differences from the 
PACE group Is in doubt* 
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PACE is not an Adult Basic Education program alone, ^"^^^^^^f ^"""^ggg^ 
combine adult and child services into one program (Heberle 1988, 
igll) comparison of PACE with ABE programs fails to evaluate the 

^««-iT^nm«nt factors that perpetuate this cycle. While PACt. xneory 
wSui™gest ?ha? Obtaining a^GED is an important goal of the program 
to be Pursued in its own right, program services could J«sult in 
realization of the program's long term objectives even if PACE adult 
s?uden?s do no bet?e? than o^her %BE students. For this reason we must 
eva?ua^e thS effects of the early childhood education component and 
?he interaction of the various services provided by the program. 

Pvfliuation of these latter two aspects of the program is not as simple 
areva^ia?in§ ^hrsuccess of the adult e<^--^f ^^f :rS?arpr^^^ 
the effects of early childhood aer^-ices and of the total P'^?^ , 
for a longitudinal design since the program is based on the belief 
thftt services should result in long-term effects on the children and 
families who participate. Ideally the PACE staff should Judge the 
p?:^rar on the basis of twenty years or more of evaluation as was 
dotS bv the Perry Preschool Project. An ideal design would also 
^nctude"" a non^r^tment control group and at least one control group 
that was provided an alternative form of treatment. 

The need for such a complex evaluation over so long a period is 
problematic? Funding sources usually prefer to know fairly soon 
Shether a program is working so that problems can be solved or so the 
^unds can be reallocated to some other program. Fortunately shorter- 
te^^m indicators can tell us whether PACE is having an effect. These 
^nSca^ors relate to the known relationship between school performance 
even "n the early years - and the li^^elihood that an individual 
will drop out of school. 

individuals who are likely to drop out of school can be identified as 
earlv as ?he primary grades. In a study of school dropouts in Boone 
Countv west V?rginil, Williams (1985) found that dropouts were absent 
from ""sch^^ many mor^ day. per year than graduates beginning in the 
first crade and continuing through the eleventh grade. LlOY^ (1978) 
established ?hat dropouts can be predicted with a fair amount of 
certainty from third grade school records. The most P^^^ictive 
variables were retention in one or more grades, number of siblings, 
^< n^- flveraae and readir>3 achievement (multiple R» . 509 ) . 
Itroup Sd Robins (1972) in a retrospective study of black children 
Stroup ^^,at grade retentions, number of absences, 

invo!vemeSi' ;it™ lega^ system, and reading achievement were good 
predictors of dropout status. 
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It is Important to stress that longitudinal research on the PACE 
program Is essential to demonstrating Its cost-benefit and that this 
research ought to be undertaken. In the shorter-term a research 
strategy that evaluates the Impact of PACE on children In the primary 
school grades will provide appropriate measures of the program's 
benefits. Follow-up of children on these measures has already begun 
and should continue, but follow-up of the children and their parents 
without the use of a control group will not provide a valid measure of 
the program's performance or a basis for estimation of program cost- 
benefit. 



A demonstration of PACE'S effectiveness must show that outcomes occur 
that would not occur if the same individuals were not involved in 
PACE. This requires that we have a comparison of the success in 
education of children similar to PACE children who do not attend PACE. 

An attractive approach is Campbell and Stanley's (1963) "nonequivaient 
control group" design which assembles a control group on the basis of 
some assumption about their similarity to the experimental group. 
The PACE program allows the creation of this type of group because of 
the available measured information. Each of the PACE participants in 
the 1987 Kim study completed the PACE participant questionnaire which 
Includes quite a bit of demographic information, and all PACE 
participants could be pre- and posttested using the Tests of Adult 
Basic Education (TABE) as are all ABE students. Administration of the 
PACE questionnaire to non-PACE ABE participants would give enough data 
along with TABE information to demonstrate the similarity of a control 
group of adults to the PACE adults. Use of similar measures of 
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children's preprogram performance would demonstrate the similarity of 
a group of children enrolled in some other non-PACE early - childhood 
program, and children who attend no EC program, to PACE children. 

The PACE program introduced the following factors which might have 
caused a program effect: 



1. 


the 


nonspecific effect of attention from helping persons; 


2. 


the 


effect of Adult Basic Education instruction; 


3. 


the 


nonspecific effect of child care; 


4. 


the 


general effect of early childhood education; 


5. 


the 


effect of the High/Scope materials; 


6. 


the 


effect of training in parenting skills; 


7. 


the 


effect of providing transportation; 


8. 


the 


effect of recruitment and selection; 


9. 


the 


effect of simultaneous services to parents and children; 


10, 


the effect of teaching and management strategies at specific 
program sites; 



11. interaction effects among all of the above. 

A pure research design could sort-out the effects of all of these but 
would be very expensive and might not even be possible. Constructing 
all of the groups with enough subjects to completely cross all of 
these effects would involve an enormous number of participants. 

Our inability to perform a methodologically flawless design should not 
deter us from performing as good an evaluation as circumstances will 
allow. A first step in an adequate evaluation study is the 
construction of an explanatory causal model of PACE services (Bryk 
& Raudenbush, 1988). This model would assure that we had been 
thorough In our design of evaluation components and would allow us to 
design mathematical models that have the greatest explanatory power 
for the amount of money and effort expended. 
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The following is proposed as a general strategy to be elaborated after 
a causal model has been developed: 

A. Use a nonequivalent control group design to test the main 
effect — i.e., whether the PACE model without consideration 
of the specific causes improves the educational performance 
of disadvantaged children and whether it gets results 
equal or superior to other programs available to its 
clientele. Estimate cost-benefit and cost-efficiency on the 
basis of this study. 

B. Use smaller studies to test the simple effects of some of 
the components of the program, such as child care and 
transportation . 

To evaluate the PACE program we shou3u test the following hypotheses: 

HI: Positive changes in parental basic skills will improve children's 
school performance. 

A test of this hypothesis requires that there be pre- and posttest 
measures of basic skills for adult PACE participants. its proof 
requires that positive changes in basic skills be associated with 
changes in children's school performance. It is not necessary to show 
that the PACE program is more effective than other interventions that 
Improve parents* basic skills, but it is necessary to show that 
children of parents whose basic skills improve make better educational 
progress than similar children whose parents' basic skills do not 
Improve . 

H2: Parental modeling of positive attitudes and behaviors about 
education will Improve children's educational performance. 

Direct measurement of parental modeling of attitudes and behaviors 
requires observational assessment on several occasions. The PACE 
program defines obtaining a GED as modeling behavior but this is 
really just an indicator that an attltudinal change has already 
occurred. Direct ecological measurement of attltudinal change — as 
opposed to indirect measurement of an Indicator like getting a GED — 
Is not excessively expensive or difficult. Kim clearly had some sort 
of direct measurement of this kind of behavior In mind when she asked 
parents how often they read to their children. ScHne studies of early 
childhood programs have actually measured this type of behavior in the 
participants' homes (Jensen, 1969; Lally, Mangione, and Honig, 1987). 
CcMnparlson of the differential performance of otherwise similar 
children whose parents did or did not obtain a GED should partially 
evaluate this hypothesis, but the addition of ecological measures 
would improve the evaluation. At minimum, it would be possible to 
collect observations of parent and child interactions during PACE 
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sessions. If there is a differential effect for children of PACE 
participants versus those of equivalent ABE participants then the 
superiority of the PACE model has been established. 

H3: Positive changes in parenting skills will improve children's 
educational performance. 

Parenting skills training is nested in PACE/ABE participation status. 
ABE programs do not teach parenting skills and it is doubtful that a 
nonequivalent control group of individuals who have had this service 
from some outside source could be easily assarabled. PACE might 
conduct a small study that trains non-PACE ABE participants in 
parenting skills. ABE students who were not taught these skills would 
be essential as a non-treatment control group. 

It is important that we develop a model of what particular parenting 
skills are to be evaluated and improved before we begin a test of this 
hypothesis. This was a serious defect in the previous studies' 
approach in this area. The literature review above should give us a 
good description of those parental behaviors that are important for 
child educational achievement. 

H4: For children for whom it is an appropriate alternative, early 
childhood education within a family literacy program will be a 
r.ore effective alternative than other programs intended to 
improve the educational performance of children. 

Some early childhood education programs have been shown to reduce 
dropout risk and to improve all of the factors that predict school 
dropout. Early childhood services are available from a number of 
programs including Headstart, the Office of Education for Exceptional 
Children, local Mental Health-Mental Retardation Boards, and private 
organizations. Part of the process of demonstrating the cost-benefit 
of PACE is demonstrating that for those children for whom it is 
appropriate, PACE works better. This requires two control groups of 
children who are similar to PACE children. One group should receive 
some other type of early childhood education and the other group 
should receive both early childhood and have parents who are receiving 
ABE services. 

It would be helpful In general to enumerate the number of alternatives 
available to PACE children before making these ccnnparisons. It also 
is very Important to establish exactly which population and exactly 
what gaols are equivalent across these programs. 
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H5: Services delivered during a critical preschool time period will 
have a long-term effect on an individual's school performance. 

Evaluation of the PACE program must include evaluation of this 
hypothesis and the one that follows. PACE treatment occurs only when 
children are in a narrow age range, which Implies that the effect of 
services is likely to be greatest when provided to a child in this 
range. We must eliminate c«npeting hypotheses relating to the age and 
duration of remediation if this key element of the pace 
prevention/remediation model is to be evaluated. Comparing the 
performance of PACE children to a nonCKjulvalent control group of 
children who received learning skill remediation (preferably with the 
High/Scope materials) at a different age would evaluate this 
hypothesis. 

H6: A combination of services provided simultaneously to parents and 
children will be more effective than the presentation of the same 
services separately by different service providers on different 
occasions . 

This hypothesis allows us to test the effect of child care, 
transportation, and joint parent-child interaction and learning and 
ideally would Involve the evaluation of the interaction effect as 
well, i.e., would involve examining groups who had these services 
separately and in combination. I doubt that It would be possible for 
us to conduct this Ideal test since 1^ is unlikely that parents of 
young children could attend basic educ& ion programs at all without 
some form of transportation and child care. However we could test the 
simple effect of child care provided by the PACE program compared to 
child care obtained independently by participants of ABE programs, 
crossed with transportation and joint parent/child learning. We could 
also evaluate the effect of in-home vs. school-based services. 

H7: Mothers who participate in PACE projects will increase the amount 
of time they spend doing the following and will continue these 
behaviors after program completion: 

reading to their children; 

directly involving themselves In school activities; 
talking with/listening to children. 

Assessing this requires pre- and posttest measures of parents' 
behaviors in these areas. Kim attempted to measure these with 
questionnaires with at best equivocal results. Actual measurement of 
the behaviors Is possible and would be helpful. 
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ERIC 



Appendix A 



A note about the use of change scores 



Educational programs frequently use the difference between pre- and 
posttest scores as a measure of program performance. These "change 
scores" have intuitive appeal as indicators of how much gain has been 
made by students. They should not be interpreted without caution, 
however, because of a couple of problems. 

Change scores by their nature are unreliable, and the amount of 
unreliability is directly related to the strength of the relationship 
between the pre- and posttest scores (Cronbach, 1970; Linn, 1988). 
As aggregate measures of program performance they are acceptable, but 
as measures of the gains made by Individuals they should be viewed 
with caution. 

There is a more serious problem with change scores in the PACE 
program. Table 8 shows the crosstabluatlon of reason for program exit 
with whether or not posttest reading scores were on an individual's 
file in the Adult Education database. This table resulted in a chi- 
square statistic of 46.48. With 2 degrees of freedom, this is 
significant at the .001 level. The implications of this result are 
that an individual who exits unsuccessfully from the PACE program is 
less likely to have his or her scores represented in the database. 
This implies that the mean posttest scores are probably biased 
upwards, since Individuals who leave before program completion can be 
expected to have learned less than individuals who completed the 
program . 



Table 8. 



Exit reason 



reading score 
present 



reading score 
absent 



Achieved goals 

End of program year 

noncompletion 



71 
52 
17 



26 
76 
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